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An appar atus and a method for the detection of objects 

The invention relates to an apparatus and a method for the detection of an 
object moving in the monitoring range of a camera. 

A reference image with an object-free monitored region is usually recorded 
and stored with such methods or apparatuses. Images of the monitored 
region are then continuously recorded and compared with the stored 
reference image during the actually occurring monitoring- If deviations 
result here, the presence of an object is assumed and a corresponding 
reaction triggered, for example a switching off of a machine. The 
comparison of the recorded images with the stored reference image takes 
place, for example, in such a way that the brightness of pixels of the 
reference image and of the actually recorded image which mutually 
correspond with respect to their spatial position are compared to one 
another and then a reaction is triggered if the determined brightness of a 
specific number of pixels of the actually recorded image does not agree 
with the corresponding brightness values of the pixels of the reference 
image. 

A problem in the use of known apparatuses and known methods of the 
said kind is the fact that the camera and/or parts of the monitored region 
can move, for example as a consequence of permissible vibrations or 
oscillations, which can then result in changes in the images recorded and, 
with an object-free monitored region, in an unwanted triggering of 
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reactions. Being more specific, it is for example possible for a hazardous 
region of a machine to be monitored using the camera with the intention 
being to switch off the machine whenever a person moves into the 
monitored region. Such a person naturally causes brightness differences 
5 in the images recorded during the monitoring with respect to an object- 
free reference image so that when persons intrude into the monitored 
region, a stopping of the machine can take place in a desired manner. If, 
however, a vehicle now drives past the monitored machine, for example, 
and produces vibrations which move either the camera or the machine 
10 slightly, an - unwanted - stopping of the machine will also take place in 
this case due to the differences between the images recorded and the 
reference image, even though no person is located in the monitored region. 

An object of the invention is to further develop a method or an apparatus 
15 of the kind initially mentioned such that, on the one hand, an 

unintentional triggering of reactions is prevented and such that, on the 
other hand, it is ensured at the same time that the reliable triggering of a 
reaction is guaranteed on the intrusion of objects or persons into the 
monitored region. 

20 

This object is satisfied in accordance with the method of the invention by 

- an actual image of the monitored region being recorded by the camera; 

- at least one actual measured value being derived from the actually 
recorded image which provides information on differences between at 

25 least two different image regions and which is invariant with respect to 
image displacements, image rotations and/or image size changes; 
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- the actual measured value being compared with a corresponding stored 
reference value derived from a reference image recorded by the camera; 
and 

- an object recognition reaction being triggered on a pre-set deviation of 
5 the actual measured value from the reference value. 

This object is satisfied in accordance with the apparatus of the invention 
by 

- a camera for the recording of an image of the monitored region; 

10 - a device for the derivation of at least one actual measured value from 
the actually recorded image, with the measured value supplying 
information on differences between at least two different image regions 
and being invariant with respect to image displacements, image 
rotations and/ or image size changes; 

15 - a comparison device for the comparison of this actual measured value 
with a corresponding reference value derived from a stored reference 
image recorded by the camera; and 

- an object recognition stage for the triggering of an object recognition 
reaction on the finding of a pre-set deviation of the actual measured 

20 value from the reference value. 

In accordance with the invention, therefore, each pixel of a recorded image 
is not compared to the corresponding pixel of the reference image as in the 
prior art, but instead reference values or measured values are determined 
25 both from the reference image and from the actually recorded images 

which only supply information on differences between different regions of 
the respective images. Only relative information relating to a reference 
image or an actually recorded image is therefore examined in each case 
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and not absolute information such as the brightness of a specific pixel as 
in the prior art. This relative information of an actually recorded image 
determined in accordance with the invention is then compared with the 
corresponding relative information of the reference image. Only when 
5 sufficiently large differences are found here, is a reaction triggered. The 
said relative information has the advantage with respect to absolute 
information that it can be invariant with respect to image displacements, 
image rotations and/ or image size changes so that, for example, vibrations 
of the camera or of the monitored region do not result in a change in the 
10 relative information or of the determined measured value. 

A structure is preferably superimposed on the reference image and the 
actually recorded image which is an element of the image. The reference 
values and the measured values are then gained from the corresponding 

15 structure information. 

The object-free monitored region can, for example, be provided with a grid 
structure which is then recognized as structure information, whereupon, 
for example, the mutual spacings of different grid lines are determined as 
the reference values or measured values in accordance with the invention. 

20 Such measured values and reference values, shown here only by way of 
example, are invariant with respect to image displacements and image 
rotations in accordance with the invention, which has the consequence 
that these measured values do not change, if the monitoring camera is, for 
example, set into vibration. Since the measured values therefore do not 

25 change with respect to the reference values in such a case in which only 
the camera was set into vibration and no object is present in the 
monitored region, a reaction is accordingly not triggered either in the 
desired manner. If, however, an object moves into the monitoring region, 
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this object will either cover a grid structure fixedly secured in the 
monitoring range or the grid lines of a grid structure projected into the 
monitored region will be displaced by different amounts. In each case this 
then results in a change in the measured values that are determined and 
5 accordingly also in a desired triggering of a reaction, for example a 
switching off of a machine. 

As already mentioned, the structure in the monitored region can be 
generated during the recording of the reference image and of the actual 
10 images of the monitored region by means of a projection device; equally, 
however, also by means of a moved light beam, for example of a laser 
beam. This has the advantage that the monitored region does not need to 
be manipulated in any way, for example by gluing on a structure foil. 

15 It is furthermore of advantage if - simultaneously or in time sequence - 
mutually different structures are generated in the monitored region since 
then the method of the invention can be carried out with respect to each 
individual structure, which reduces the risk of erroneous reactions being 
triggered or of reactions being erroneously omitted. In this respect, 

20 different projection directions or different devices for the generation of the 
moved light beam can be used for the generation of the different 
structures; equally, however, it is also possible to generate different 
structures by means of a single device. 

25 In accordance with the invention, a reaction is preferably only triggered 
when the comparison of the actual measured value with the reference 
value provides a comparison value which lies outside a pre-set tolerance 
range. This additionally reduces the risk for erroneous triggerings since 
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differences between the measured, value and the reference value lying 
within the tolerance range do not trigger any reactions. 

It is of advantage if the image information or structure information 
5 comprise, among other things, information with relation to the reference 
points, with the reference points in particular marking the boundaries of 
the monitored region. These reference points facilitate the locating of the 
structure information and reduce the complexity of the required image 
recognition algorithms. Reflections can be used for the marking of the 

10 reference points which are arranged at invariant positions relative to the 
monitored region and, for example, mark the edge of the monitored region- 
It is particularly preferred if a plurality of actual measured values of the 
same or of different type are derived from the actual image information or 

IS structure information and is compared with corresponding stored 
reference values derived from the image information or structure 
information of the reference image, The risk of erroneous reactions or 
erroneously omitted reactions can additionally be reduced by the use of a 
plurality of measured values, 

20 

The measured values and/ or the reference values can preferably include 
one or more of the pieces of the information given below: 

- distance between two different image regions or structure regions or 
25 between a reference point and an image region or a structure region; 

- brightness difference between two different image regions or structure 
regions or between a reference point and an image region or a structure 
region; 
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- color difference between two different image regions or structure 
regions or between a reference point and an image region or a structure 
region; 

- brightness gradient between two different image regions or structure 

5 regions or between a reference point and an image region or a structure 
region- 
All the above-named information has the common feature that it only 
represents relative information and not absolute information. 

10 

It is of advantage if> alternatively or additionally, a correlation function is 
calculated between at least one region of the actual image information or 
structure information and a corresponding region of the image information 
or structure information of the reference image and if the shape of the 
15 correlation function and/ or certain values of this correlation function are 
used for the decision with respect to the triggering of an object recognition 
reaction. This variant makes use of the advantage that a correlation 
function also ultimately only represents relative information which is not 
influenced, for example, by displacements or rotations of image regions. 

20 

The monitored region is preferably illuminated by means of at least one 
light source during the recording of the reference image and of the actual 
images of the monitored region. This allows the camera to detect further 
processable images without problem. 

25 

The working order of the camera used can be checked by recording images 
regularly with and without a structure generated in the monitored region 
and subsequently checking whether the structure information is found 
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when a structure is generated and is not found when a structure is not 
generated. In this respect, mutually different structures can preferably be 
used for the checking of the camera which are generated in the monitoring 
range simultaneously or in time sequence. The said checking of the 
camera must be carried out in this alternative at a point in time at which 
no monitoring function takes place or at which the monitoring function 
can be briefly interrupted. 

It is alternatively possible, both in the recording of reference images and in 
the recording of actual images to take always one image with a generated 
structure and one image with no structure generated in each shot and 
subsequently to process only the difference image resulting from these two 
images. In this case, an "object recognition reaction" is always triggered 
when a deviation results between the reference difference image and an 
actually recorded difference image. This deviation is then explained either 
by the presence of an object in the monitored region or by faulty operation 
of the camera so that, for example, a switching off of a machine can be 
ensured in the two hazard cases of an object in the monitored region or of 
faulty operation of the camera. 

Further preferred embodiments of the invention are set forth in the 
dependent claims. 

The invention is described in the following by way of embodiments and 
with reference to the drawings, in which are shown: 
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Fig. 1 a flowchart of the routines of the method of the invention for 

the detection of an object moving in the monitored region of a 
camera; 



5 Fig. 2 a flow diagram of the routines in the determination of the 

reference value made use of in accordance with the invention; 

Pig. 3 the schematic illustration of an apparatus in accordance with 

the invention for the carrying out of the method of the 
10 invention; and 

Figs. 4a - c different shapes of structure information which are gained 
from actually recorded images and from which measured 
values in accordance with the invention can be derived. 
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A flowchart is shown in Fig. 1 which illustrates the individual method 
steps which are carried out in the method of the invention for the 
detection of an object moving in the monitored region of a camera. 



20 The initial or starting point of the method is designated with 1. 



At the start an initialisation of the total system is made in step 2 > which 
comprises the resetting of variables to their start values, the emptying of 
memories in an evaluation unit and putting the camera into a state ready 
25 for recording, etc. This is not explained in further detail at this point since 
such initialization procedures are state of the art and a matter of course 
for one skilled in the art. 
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At step 3, an actual image of a monitored region is recorded with a 
camera, which means that a virtual image consisting of pixels of the 
monitored region is generated by any desired image sensor and that said 
image is electronically stored in an image memory. In the simplest case, 
5 the pixels can be represented in binary manner, that is black or white > 
have different gray stages or have colors with different color gradations. 

Structure recognition is carried out in step 4 by a recognition device of the 
evaluation unit. The recognition device uses an algorithm for this purpose 
10 which is able to recognize structures such as line regions, grid regions or 
areal regions from the image stored in an image memory, that is to make a 
mathematical correlation between pixels which represent an object or a 
part of an object. 

15 At step 5, a derivation device of the evaluation unit can derive a measured 
value from such a structure which does not change, for example, with a 
pre-set, permitted displacement or rotation of the structure. 
With structure information including a grid structure, such a measured 
value can comprise the spacing of adjacent grid lines, the angle at which 

20 the grid lines intersect, the area enclosed by specific grid lines, etc. 

If the structure information comprises, for example, an area or an areal 
segment, the measured value can, for example, be equal to the surface 
integral. 

25 At step 6, the determined measured value is then compared by a 
comparator with a reference value calculated from a corresponding 
structure of a reference image, said reference value being determined in 
the same way as the actual measured values and being derived from an 
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image of an object-free monitored region. If the comparison of the 
measured value with the reference value results in the measured value 
exceeding a defined measure, that is a threshold value, it is branched to 
step 7 where a reaction triggering device of the evaluation unit triggers a 
5 pre-defined reaction such as the emission of an alarm signal or the 

switching off of a machine. If the comparison result does not exceed the 
threshold value, the routine returns to step 3 and a new image of the 
monitored region is recorded. The method described subsequently inns 
again from the start* 

10 

Pig. 2 illustrates the routines in the determination of the reference value 
used for the determination in accordance with the invention. After the 
start at 8, a reference image of the monitored region is first recorded by 
means of the camera at step 9 without an object not belonging to the 
15 allowed or permitted scene being in the monitored region. 

Subsequently to this, at step 10, as described for step 3 at Fig. 1, a 
structure is recognized in the image supplied try the camera and stored in 
the image memory. 

20 

At step 1 1, the evaluation unit derives a value from the recognized 
structure which is invariant with respect to pre-determined image changes 
explained in connection with step 4 (Fig. 1) and stores this as the 
reference value. 

25 

Furthermore, at step 12, a threshold value is determined which represents 
a maximum permitted deviation in magnitude between an actual 
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measured value determined at a later point in time and the reference 
value, without a reaction having to be triggered by the evaluation unit. 

The method of Fig. 2 will usually run as part of step 2 in accordance with 
5 Fig. 1. 

Fig. 3 shows, in a schematic representation, a camera 13 and an 
Ulumination and projection device 14 which are both arranged in a 
common housing and whose lenses are directed to a monitored region 15. 
10 The fields of view of the lenses of the camera 13 and of the illumination 
and projection device 14 mutually overlap such that at least substantially 
the same section of the monitored region IS is detected by both devices. 
The monitored region 15 can, for example, represent the boundaries of a 
hazardous region in front of a machine. 

15 

The illumination and projection device 14 projects a structure of dark 
elements 16 located on a light background onto the monitored region 15. 
The camera 13 detects an image of the monitored region 15 with a 
structure projected thereon and supplies corresponding image information 

20 to an evaluation unit (not shown). This evaluation unit then determines 
structure information corresponding to the projected structure from this 
image information, whereupon the measured values x and y shown in Fig. 
3, for example, can be determined from this structure information. The 
measured values x and y both represent relative information which in 

25 each case does not relate to the absolute brightness of pixels of the image 
recorded, but represents only information which relates to differences 
between different image regions. The measured value x, for example, thus 
represents the mutual spacing of two dark structure regions. The 
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measured value y represents the width of a dark structure region- The two 
measured values x and y are here only mentioned by way of example; any 
further measured values such as color differences, distances to reference 
points, brightness differences, etc, can also be derived from the structure 
5 information of Fig. 3, 

Pigs. 4a to c show one-dimensional brightness plots, entered over the 
position x of a dimension of a reference image selected as an example and 
of actually recorded images with measured values derived therefrom* 
10 However, measured values can likewise be derived from two or three 
dimensional image information within the framework of the invention, 

In this respect, the brightness plot along a line of a reference image is 
shown in Fig. 4a, The reference image is recorded with an object-free 

15 monitored region and shows a structure with five bright regions located at 
mutually different distances which are each mutually bounded by dark 
regions. The reference values aj to a4 determined from the reference image 
relate to the mutual spacings of adjacent bright structure regions, 
whereas the reference values bi to b3 represent the spacings of mutually 

20 adjacent dark structure regions. 

The reference values ai to a4 and bi to b$ are stored by an evaluation unit 
in order to be able to be successfully compared with actually determined 
measured values in monitoring operation. 

25 

Fig. 4b shows the brightness shape along a line of an image actually 
recorded during the monitoring operation, with a - permitted - 
displacement having taken place between the camera 13 and the 
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monitored region 15 (see Fig. 3) during the recording of this actual image. 
This displacement results in the fact that the total image recorded was 
likewise recorded in displaced manner, which accordingly results in a 
displacement of the total brightness shape in accordance with Pig. 4b with 
5 respect to Fig. 4a. In this respect, however, all bright and dark regions are 
displaced in the same manner so that the actual measured values a* x - a' 4 
and b'i to b'3 determined in accordance with Fig. 4b each remain 
unchanged with respect to the reference values ax to a*4 and bi to fc>3 
determined in accordance with Fig. 4a. Since no deviation can thus be 
10 found between the measured values and the reference values, no reaction 
is triggered either. 



However, if an unpermitted object is now moved into the monitored region, 
it is possible that the projection location of a bright region is displaced, 
15 which is illustrated by way of example in Fig. 4 by a dashed line. This 

displacement then results in a change in the measured values a' 2 , a" 3 , b'i 
and b'3- These actual measured values then no longer correspond to the 
reference values associated with them so that a reaction is triggered in a 
desired manner. 

20 

Figs. 4a and 4b thus show that a displacement between the camera and 
the monitored region does not on the whole result in a reaction triggering, 
but that a reaction is only triggered when an unpermitted object is 
actually present in the monitored region. The same applies accordingly 
25 when a variable interference light level is superimposed on the actually 
recorded image, as is shown by way of example in Fig. 4c. 



It can be seen from Fig. 4c that the distances and widths of the bright and 
dark structure regions do not change with respect to Fig. 4b if a variable 
interference signal is superimposed on the actually recorded image. The 
sum signal of wanted signal to unwanted signal represented in Fig. 4c also 
allows the distances between the bright regions and between the dark 
region to be determined without problem and accordingly allows the 
measured values to be calculated* These measured values a M, i to a" * 4 and 
b" i to b M 3 are identical to the measured values of Fig. 4b so that no 
reaction is triggered with the plot shown in Fig. 4c either. 

If, however, a displacement of the central brightness region (which is in 
turn shown by a broken line in Fig. 4c) occurs as a consequence of an 
object intruding into the monitored region, then the measured values 
change as well, as was described in connection with Fig. 4b, Accordingly, 
a reaction is then also triggered. 

The use of the distances between bright regions and dark regions is in 
turn only shown by way of example in Fig. 4; any other measured values 
can likewise be derived from relative information between different image 
regions so that the advantages of the invention can also be achieved with 
these other measured values. 



